1. Field of the Invention
The present invention relates to an image processor that has a flexible zoom function for generating an image, of which the resolution is higher than that of a single pixel of the original image that can be captured with an image sensor.
2. Description of the Related Art
With remarkable development of digital video equipment, the image sensors of most consumer digital still cameras now have a resolution of 1 megapixels. As a result, as far as still pictures are concerned, a resolution that is high enough to print them as beautiful photos has already been achieved. In the following applications, however, the resolution should be further increased beyond their image capturing limit:                (i) when a region of interest, which is a portion of a product that is offered for sale in the field of online shopping, for example, needs to be zoomed in on to an arbitrary zoom power;        (ii) when a diseased part of a patient needs to be inspected closely by medical imaging;        (iii) when the clothes of a culprit need to be identified in detail with a surveillance camera; and        (iv) when a face, iris or fingerprint image needs to be analyzed in the field of biometrics.        
Attempts to meet these demands by increasing the resolution of image sensors have recently almost reached a limit, because an optical limit prevents the resolution from being increased anymore even if the size of a pixel is decreased. To go over that limit, there is no choice but to increase the size of a single pixel for a lens or an image sensor. As a result, it has become more and more difficult these days to realize a camera of an even smaller size for use as a digital camera, a digital movie camera, a small surveillance camera, or a medical endoscope.
There is an attempt to generate a high-resolution image by dividing a single pixel with a conventional intensity even finely while using a small camera. Such an attempt is generally called a “super-resolution technology” for increasing the resolution of an image. According to this technology, a number of images, which have been obtained by shooting either a number of still pictures of a subject sequentially or a moving picture of the subject, are used as an input. Then, the correspondence between an associated group of pixels, which represents the same portion of the object, is defined between those images, thereby estimating the pattern on a high-resolution image based on such an associated group of pixels. However, normally only one still picture is used as a photo representing a product that is offered for sale in online shopping or as a medical image. Also, although video captured by a surveillance camera is a moving picture, discrete still pictures are often used to form the moving picture. That is why the conventional “motion-based” resolution increasing technique is not applicable to these situations.
Also, according to the conventional technique, the magnitude of motion between corresponding portions of an object that are included in a number of image frames at different times is detected by an intensity-based association technique. For that reason, as for an area where a lot of specular reflection is produced at the surface (e.g., on a glossy surface), it is difficult to detect the magnitude of motion.
A technique for dividing a single pixel finely even on such a glossy surface on the supposition that a still picture is used is disclosed in Ping Tan et. al, “Resolution-Enhanced Photometric Stereo”, European Conference on Computer Vision, 3: pp. 58-71, 2006 (Hereinafter, Non-Patent Document No. 1). According to that technique, an object (such as a metallic coin or a stone engraving), of which the specular reflector meets a microfacet physical reflection model, is selected. A lot of object images are shot using a fixed camera with the direction of the light source changed in various patterns. Then, by applying those images to a microfacet model, the spatial distribution of microscopic normals within a single pixel is estimated based on those many images and it is calculated how high the intensity would be when the microscopic shape within a single pixel is illuminated with the light. In the first half of this resolution increasing technique, a histogram of normals to microscopic regions that are distributed within a single pixel is formed by making a lot of shots with the light source moved. In the latter half, on the other hand, the combination of the spatial arrangements of microscopic region normals that form such a histogram is solved as an energy minimizing problem that is mainly based on the integrability of normals. As used herein, the “integrability of normals” means that with a closed path given to the surface of an object, when surface height information, which is a linearly integrated value of the normals on that path, makes a round of the path and returns to the original position, that surface height becomes the same as the original height at that position.
A technique for increasing the resolution of an image by capturing a number of images with the light source moved is disclosed in Manjunath et. al, “Simultaneous Estimation of Super-Resolved Depth Map and Intensity Field Using Photometric Cue”, Computer Vision and Image Understanding 101(2006), pp. 31-44 (Hereinafter, Non-Patent Document No. 2). According to this technique, objects that can be total diffuse reflection models (such as a stuffed doll or a pair of sneakers) are selected, and 8 to 16 images are shot, thereby increasing the resolution of 2×2 images. This technique uses normal images, albedo image resolution decreased models, normal integrability constraints, and Markov random field probability models, thereby increasing the resolution of the object normals and the albedo reflectance at the same time.
Japanese Patent Application Laid-Open Publication No. 2007-86720 (Hereinafter, Patent Document No. 1) discloses a patterned polarizer. On the other hand, Japanese Patent Application Laid-Open Publication No. 11-313242 (Hereinafter, Patent Document No. 2) and Lawrence B. Wolff, “Polarization Vision: a New Sensory Approach to Image Understanding”, Image and Computing 15 (1997) (Hereinafter, Non-Patent Document No. 3), pp. 81-93 disclose a device, of which the polarization plane can be rotated. The elements disclosed in those documents can be used in preferred embodiments of the present invention to be described later.